1. Field of the Invention
The present Invention is generally related to systems and methods of managing communications by electronic mail and, in particular, to a method and system for detecting and blocking the receipt of unsolicited commercial electronic mail.
2. Description of the Related Art
With the expansion of the Internet as a medium for the transport of electronic mail (e-mail), an advertising practice of sending unsolicited commercial e-mail (UCE or “spam”) has developed as an annoyance to other uses and users of the Internet. Indeed, the volume of UCE received by an e-mail recipient may regularly be a substantial percentage of all e-mail received.
Bulk e-mailers, as the sources of UCE are commonly referred to, utilize the resources of the Internet itself, including specifically, the Usenet news network, Web based discussion groups, Web based indices of users and organizations, and other public or pseudo-public information sources to gather e-mail addresses. E-mail address lists are also available for purchase from various organizations who at least purport to provide targeted lists. The quality of targeted lists, however, tends to degrade rather quickly over time due to the fluid nature of the Internet and the changing interests of Internet users. Consequently, there is an obvious interest by the bulk e-mailers to oversubscribe their mailing lists with any and all e-mail addresses that are possibly relevant targets for the content of any particular UCE.
There is also little commercial constraint limiting the lowering of the relevancy threshold used by bulk e-mailers due to the beneficial cost structure that most bulk e-mailers enjoy. The only actual cost to the bulk e-mailer for sending additional UCE is, at most, the incremental cost of acquiring additional e-mail addresses and the incremental connection cost required to send a UCE message. Many higher bandwidth Internet connections are structured for continuous availability at a fixed fee. Bulk e-mailers with these types of connections therefore effectively incur no additional cost for sending UCE to as many e-mail addresses they have acquired.
Fortunately, there is at least one significant non-economic, or at least not directly economic, cost-of-business faced by bulk e-mailers. Internet access for all users is channeled through Internet Service Providers (ISPs) at some level. As predominantly service oriented organizations, these ISPs are usually responsive to complaints from the general Internet public concerning excessive UCE being sent from any domain serviced by a particular ISP or from a subscriber ISP site served by a higher level ISP. Often, ISP service agreements include contractual prohibitions restricting their subscriber sites and user domains from sending UCE.
Bulk e-mailers, however, have responded by utilizing various techniques to obscure the source of the UCE they send in order to avoid the identification that would lead to limits on their activities. Conventionally, Internet e-mail messages include a header section that includes multiple required and optional lines of information including the source and destination addresses of an e-mail message. Typical required lines include From, To, Message-ID, and X-Mailer. Optional lines include Reply-To, Organization, and Return-Path. Additional required header lines, denominated Received, are added to the message as the message passes through the mailer sub-systems of ISPs and other computer systems as necessary to reach a destination domain user. These Received lines are nominally beyond the control of individual bulk e-mailers. The definition and use of these header fields is specified in Request for Comments (RFCs)/Standards (STDs) documents 821/10, 822/11, and 974/14, among others, which are publicly available from multiple sources on the Internet.
Bulk e-mailers use anonymous mailers and re-mailers to obscure the required e-mail header lines of their UCE by specifying non-existent e-mail systems and accounts. Modified mailers can be used to completely remove header lines or to substitute addresses of known valid Internet e-mail accounts that are not actually associated with the bulk e-mailer.
These techniques are generally sufficient to prevent the UCE recipients from being able to identify and complain to the postmaster of the relevant ISP about the activities of a bulk e-mailer. While the Received lines provide traceable information, the complexity of filtering through this information is usually beyond the level of effort that most UCE recipients will undertake. Even for those that do, the number of public complaints actually received by the ISP is significantly reduced, often allowing the bulk e-mailer to remain in operation, if only for a longer period of time before being forced to find a new ISP to use for their activities.
A variety of techniques have been developed in the recent past to deal with the growing amount of UCE being received by Internet e-mail users. These techniques primarily include e-mail client systems supporting manual e-mail accept and reject lists, automated context analysis, use of public shared lists of known spam sources, and direct challenge systems. None are completely effective and all impose an additional degree of operating complexity on the e-mail client user to varying degrees.
The typical e-mail accept and reject list approach, as used in the ProntoMail® e-mail client, provides for lists of e-mail addresses that are used as gate filters against incoming e-mail. E-mail with addresses on the accept list are passed, while messages with addresses on the reject list are discarded; the reject list is a blocking filter. By default, e-mail addresses not on either list are presented to the user to determine which list to add the address to. As a result, the user sees all of the UCE that originates from any address that is new to the user's e-mail client. Since the bulk e-mailers to hide or frequently change their return e-mail addresses, almost all of the UCE is seen by the e-mail client user.
Automated context analysis, such as implemented by DeadLetter®, a Eudora® plug-in, relies on key word usage and various patterns of advertisement pitches to discern UCE from other e-mail. Suspected UCE is automatically discarded or, more typically, directed to an alternate e-mail in-box of the client. Since the analysis is not and as a practical matter cannot be perfect, desired e-mail may be mis-characterized. Therefore, the user is generally required to manually review the messages in the alternate e-mail in-box anyway. E-mail mis-characterized such that it is unrecognized as UCE winds up in the standard in-box. Thus, this technique functions only as an imperfect segregating filter against UCE, rather than a blocking filter.
Public shared lists, manually developed and currently maintained by only a few large organizations, such as AOL®, can be automatically referenced by modified e-mail clients. These lists are used as simple reject lists, thus supporting the implementation of blocking filters. Unfortunately, large organizations are required as a practical matter to develop and maintain these lists. There is no guarantee that these organizations will continue to publically disseminate these lists, particularly in view of the cost of the preparation and maintenance of the lists. Even so, these lists are almost by definition out of date due to the necessary delay in their compilation. In contrast, the techniques of the bulk e-mailers tend to rapidly obsolete the lists.
The foregoing filtering systems are basically passive or receptive in nature. These systems operate to actively filter e-mail as received, but they are passive with regard to using the larger Internet to provide some basis or support for implementing their blocking filters. A challenge system, such as used by the MailGuard® e-mail client, “actively” issues an e-mail response to any e-mail received from an address that is not identified on an accept list. That is, the challenge system typically maintains and uses accept and reject lists similar to ProntoMail®, but adds the automatic challenge issuance for mail from unknown addresses. If the response to the challenge e-mail is acceptable, as determined by the client user, the previously unknown e-mail address is added to the accept list. Thus, the challenge system, and other similarly active systems, are generally more accurate than others in discriminating between UCE and desired e-mail. This accuracy, however, comes at the price of greater client user involvement in the discrimination process. This increased involvement is viewed as both unavoidable and a substantial and undesired penalty characteristic of active challenge type systems. Consequently, the relatively “passive” blocking filter systems are conventionally viewed as far more desirable despite their recognized shortcomings.
There is therefore a clear need for an improved UCE oriented blocking filter that can be implemented by Internet e-mail clients.